A Contextual Similarity Analysis System for Detecting Original Authored and Machine Generated Scientific Abstract

Abstract

Ensuring academic integrity has become increasingly challenging due to the rapid growth of artificial intelligence, as machine-generated scientific abstracts closely resemble human-written content. To address this, this study presents a Contextual Similarity Analysis System designed to detect and classify original-authored and machine-generated scientific abstracts. Traditional text classification systems often struggle with capturing deep contextual relationships and handling complex linguistic patterns, resulting in reduced accuracy. To overcome these limitations, we propose a three-module architecture. The first module utilizes Natural Language Processing (NLP) techniques such as text cleaning, normalization, tokenization, and dataset balancing to prepare high-quality input data. The second module employs transformer-based models including DistilBERT and BERT combined with Convolutional Neural Networks (CNN) to generate contextual embeddings and extract meaningful features. The final module integrates advanced contextual modeling using RoBERTa with Bidirectional Long Short-Term Memory (BiLSTM) networks along with model comparison techniques to perform accurate classification. Experimental results demonstrate that the proposed system achieves high validation accuracy, with hybrid and advanced models outperforming baseline approaches while maintaining efficient performance. The integrated web application enables low-latency, real-time classification with confidence scores, making it a reliable solution for ensuring academic integrity in research and educational environments.

Introduction

The text describes a system designed to detect whether scientific abstracts are human-written or AI-generated, addressing the growing challenge of maintaining academic integrity in the age of advanced language models.

It explains that existing NLP-based detection systems often rely on shallow or traditional feature extraction methods, which fail to capture deep semantic meaning. This leads to poor accuracy, especially because AI-generated and human-written abstracts are becoming increasingly similar in structure and style. Many current systems also lack real-time, scalable, and context-aware classification.

To solve this, the proposed system introduces a multi-module deep learning architecture that combines preprocessing, transformer-based embeddings, and hybrid neural models for more accurate detection. The input text is first cleaned and processed using standard NLP techniques, then converted into embeddings using models like DistilBERT, BERT, and RoBERTa.

The system uses three main modules:

• A baseline DistilBERT classifier for initial prediction
• A hybrid BERT + CNN model to capture both semantic meaning and local textual patterns
• A RoBERTa + BiLSTM model to learn deep contextual and sequential relationships

Finally, the system compares all models using metrics like accuracy, precision, recall, and F1-score, and selects the best-performing model to produce the final prediction with confidence scores.

The literature review highlights that while transformer models and hybrid architectures significantly improve classification accuracy, they also increase computational cost and complexity. This motivates the need for a balanced system that is both accurate and efficient.

Conclusion

This project successfully develops an intelligent system for detecting and classifying scientific abstracts as human-written or machine-generated, thereby supporting academic integrity and reliable content validation. By transforming textual input into contextual embeddings using transformer-based models and enhancing feature extraction through hybrid deep learning techniques such as CNN and BiLSTM, the system effectively captures both semantic and sequential relationships within the text. The integration of DistilBERT, BERT+CNN, and RoBERTa+BiLSTM models ensures accurate and robust classification through model comparison strategies. Achieving high validation accuracy while maintaining efficient real-time performance, the system demonstrates its effectiveness as a scalable and reliable solution for academic and research applications.

References

[1] Ali Al Bataineh, Rachel Sickler, Kerry Kurcz, Kristen Pedersen, “AI-Generated Versus Human Text: Introducing a New Dataset for Benchmarking and Analysis,” IEEE Transactions on Artificial Intelligence, Vol. 6, No. 8, pp. 2241–2247, 2025. [2] Ali Saleh Alammary,“BERT Models for Arabic Text Classification: A Systematic Review,” Applied Sciences, vol. 12, no. 11, p. 5720, June 2022. [3] Ammar Ismael Kadhim, “An Evaluation of Preprocessing Techniques for Text Classification,” International Journal of Computer Science and Information Security (IJCSIS), vol. 16, no. 6, pp. 22–32, June 2018. [4] Gang Liu and Jiabao Guo, “Bidirectional LSTM with Attention Mechanism and Convolutional Layer for Text Classification,” Neurocomputing, vol. 337, pp. 325–338, Feb. 2019. [5] Hikmat Ullah Khan, Anam Naz, Fawaz Khaled Alarfaj, and Naif Almusallam, “Identifying Artificial Intelligence-Generated Content Using the DistilBERT Transformer and NLP Techniques,” Scientific Reports, vol. 15, p. 20366, May 2025. [6] John Blake, Abu Saleh Musa Miah, Krzysztof Kredens, Jungpil Shin, “Detection of AI-Generated Texts: A Bi-LSTM and Attention-Based Approach,” IEEE Access, Vol. 13, pp. 71563–71574, 2025. [7] Joshua Mawudem Gakpetor, Martin Doe, Michael Yeboah-Sarpong Damoah, Dominic Dalyngton Damoah, John Kingsley Arthur, and Michael Tetteh Asare, “AI-Generated and Human-Written Text Detection Using DistilBERT,” Proceedings of the 2024 IEEE SmartBlock4Africa Conference, Accra, Ghana, September 30 – October 4, 2024. [8] Md Mostafizer Rahman, Ariful Islam Shiplu, Yutaka Watanobe, and Md Ashad Alam, “RoBERTa-BiLSTM: A Context-Aware Hybrid Model for Sentiment Analysis,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 9, no. 6, pp. 3788–3805, Dec. 2025. [9] Khalid M. Kahloot and Peter Ekler, “Algorithmic Splitting: A Method for Dataset Preparation,” IEEE Access, vol. 9, pp. 125229–125237, September 6, 2021. [10] Rukhma Qasim, Waqas Haider Bangyal, Mohammed Abdullah Alqarni, and Abdulwahab Ali Almazroi,“A Fine-Tuned BERT-Based Transfer Learning Approach for Text Classification,” Journal of Healthcare Engineering, vol. 2022, Article ID 3498123, Jan. 2022. [11] Shun Moriya and Chihiro Shibata,“Transfer Learning Method for Very Deep CNN for Text Classification and Methods for Its Evaluation,” in Proc. IEEE 42nd Int. Conf. Computer Software and Applications (COMPSAC), Tokyo, Japan, 2018, pp. 153–154. [12] Tugba Celikten and Aytug Onan, “Exploring Text Similarity in Human and AI-Generated Scientific Abstracts: A Comprehensive Analysis,” IEEE Access, Vol. 13, pp. aa Diyana Aldeen Ahmed, Thura Abbas, and Ayyad Rodhan Abbas, “Review of Detecting Text 74313–74322, 2025. [13] Toka A. Mohamed, Mohamed H. Khafgy, Ahmed B. Elsedawy, And Ahmed S. Ismail, ” A Proposed Model for Distinguishing Between Human- based and ChatGPT Content in Scientific Articles ,” IEEE Access Vol.12, pp. 121251-121259, 22 Auguest 2024. [14] Wei Shi, Miao Song, and Yong Wang, “Perturbation-Enhanced-Based RoBERTa Combined with BiLSTM Model for Text Classification,” Proc. ICETIS Conference, pp. 294–298, 2022. [15] Xinying Chen, Peimin Cong, and Shuo Lv,“A Long-Text Classification Method of Chinese News Based on BERT and CNN,” IEEE Access, vol. 10, pp. 34046–34057, 2022. [16] Yazdan Zandiye Vakili, Avisa Fallah, and Hedihe Sajedi, “Distilled BERT Model in Natural Language Processing,” Proceedings of the 2024 14th International Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, Iran, November 19–20, 2024. [17] Yajian Zhou, Jiale Li, Junhui Chi, Wei Tang, and Yuqi Zheng,“Set-CNN: A Text Convolutional Neural Network Based on Semantic Extension for Short Text Classification,” Knowledge-Based Systems, vol. 257, p. 109948, 2022. [18] Yunxiang Zhang and Zhuyi Rao, “n-BiLSTM: BiLSTM with n-gram Features for Text Classification,” Proc. IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 1056–1059, 2020. [19] Zixian Guo, Lu Han, and Ligu Zhu, “Research on Short Text Classification Based on RoBERTa-TextRCNN,” Proc. 2021 International Conference on Computer Information Science and Artificial Intelligence (CISAI), pp. 845–850, 2021.

Copyright

Copyright © 2026 Srilakshmi G, Monika S, Surendhar M, Dr. Ezhilarasan M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.